Just Call Me Internet

home *** CD-ROM | disk | FTP | other *** search

/ Just Call Me Internet / Just Call Me Internet.iso / docs / ascii / internet / tuning / tuning.faq

Wrap

Text File | 1997-06-20 | 58.8 KB | 1,216 lines

Archive name: ftp.demon.co.uk:/pub/doc/ka9q/Tuning.faq Posting-frequency: 14 days Last-modified: 10th July 1995 Version: 29 TUNING.FAQ ========== Changes since 23rd June 1995: * Update information on V34 access * Update information on vPoPs * Update information on news server If you have any comments on the contents of this FAQ, please let me know Changed text is marked by #s CONTENTS ======== 0. Introduction 1. Do I have a performance problem ("Am I normal ?") 2. MODEM.TXT 3. MNP5 and other compression 4. Hardware overruns 5. How fast should I run the link to my modem ? 6. Does Windows affect link speeds ? 7. Eliminate SMARTDRV, TSRs etc. 8. Serial chips 9. The KA9Q asystat command 10. Internal modems 11. IP fragmentation 12. NNTP request queues 13. HISTORY 14. PPP versus SLIP 15. Telnet speed 16. Other improvements 17. OS/2 18. The Authors 19. Disclaimer 0. Introduction == ============ This document mainly concerns itself with tuning your KA9Q system to access the Internet as efficiently as possible. Time is money (and the phone company will send you their bill eventually). Because people don't always like blindly following instructions some explanations are given as to why various things are useful. You can skip these sections if you wish and just push the buttons :) Only limited initial knowledge is assumed, but by the time you read to the end you too could be an expert. Although this document is specifically aimed at DOS and OS/2 KA9Q users, the general discussion is much more widely applicable. 1. Do I have a performance problem ("Am I normal ?") == ================================================= If you have a USR V 34 or VTerbo modem you should connect to demon via the new vPoPs. Most of the VPoPs have VTerbo modems, although these are now slowly being updated to V34 standard. Demon have also said that they hope to increase the serial port speed to 58,700 or 115,000 as this occurs. I am currently in the process of gathering figures for V34 connections. If you own a 14.4K modem which can do V42bis (such as a Sportster) and you connect to it at 38400 bps (or more) you should be getting a cps (characters per second) throughput of at least: 2700 cps for text 1650 cps for UUENCODED text 1600 cps for ZIP files By text is meant general Usenet newsgroups and email. UUENCODE is the format used for sending programs, pictures and sounds around within text-based newsgroups. ZIP is a popular pre-compressed format for binary files transferred by the FTP file transfer protocol. A 9600 bps modem with V42bis should achieve: 1850 cps for text 1050 cps for UUENCODED text 1000 cps for ZIP files A 9600 bps modem which does no compression at all should achieve a touch over 900 cps for all types of data. Obviously, if you take a mixture of newsgroups some of which are binary and some of which are not then you will get intermediate performance figures. Also if your newsfeed is mixed in with a lot of incoming email then you will get a slower news transfer while the link is being shared. You will also be lucky if you consistently achieve these speeds at very busy times (such as early weekday evenings). When there can be over six hundred people all transferring data, there is unsurprisingly, some degradation in performance. The servers are busier, and there is more contention for bandwidth on the Demon LAN. A new news machine was installed at Demon just before Christmas. It has the new Solaris 2.4 operating system and the latest NNTP software. This initailay lead to an increase in speed from the server. Thoughput is not as high as it could be though due to a bug in the Solaris operating system causing retransmittions. # Over the last few weeks the demon news server has been improved considerably so that the 400 server too busy message is rare. The problem with retransmizssions is still there though. A number of seperate syncronised news servers have also been added. The NNTP software has also been optimised by aDe. There is also some doubt that a very slow machine such as an Amstrad 1512 can ever reach these speeds, although 12MHz 286s are reported to manage just fine. Also, running OS/2 on slow 386s seems to reduce maximum news throughput. If you have a slow machine you are advised to work through the rest of this document to check that your system is optimally tuned; but you may well not achieve the suggested "normal" throughput figures. If you are using OS/2 there is some specific advice in section 17. KA9Q will show you the news transfer rate you are achieving if you have "nntp verbose on" (type this at the keyboard or add it to autoexec.net, perhaps with the DIS front end [D A f6]). The figures are also recorded in _\spool\nos.log for posterity. Even with a "perfect" set-up you will find the speed achievable for Usenet news varies from day to day, but that ZIP and UUENCODE transfer speeds are pretty constant. If you want to try some specific tests then Demon keep some special files for FTP access on ftp.demon.co.uk. You will find them in /pub/test. The most useful ones for general testing are the various "fullfile"s, since "emptyfile" and "regularfile" are easily compressed out of existence by almost everything! In fact "emptyfile" is usually so compressed across the phone link that it serves merely to measure the maximum throughput of the rest of the system. If you regularly fail to get the figures we mention then there are a number of possible reasons. You will need to check your modem is properly configured. You will need to eliminate any hardware or software overruns. Since the default KA9Q settings are wrong (!) you will probably need to tweak some parameters to eliminate IP level fragmentation. All of these changes are discussed below. If you suddenly get a slower transfer then it may be temporary. Maybe you picked a busy time, you got a noisy phone line or a system problem at Demon caused difficulty. If not a peak time, check the demon.announce newsgroup for apologies from Demon, or the demon.service newsgroup for other sufferers. It is also a good idea to finger status@gate.demon.co.uk as many minor problems are often only reported their. If it persists then it was probably the change you've just made to your machine. If it just affects news then you must check if your history file is too big (see section 13). Finally, before moving on to practical advice, please note that the figures stated are pretty much the best you can hope for (within a few percent), and assume that the only limiting factor is the link between you and Demon's Finchley network. If you are transferring data from the other side of the planet you will be competing for bandwidth across the Atlantic, through the US routers and then for some attention from a busy server in California (or wherever). Similarly, though less dramatically, when you use one of the local access tPoPs (Edinburgh, Cambridge etc.) then besides all the other factors, you will competing with the other local users for a share of the 64K line to London. Thus unless you are logging on to a modem in Finchley you may never be able to achieve quite the suggested throughput. All the vPoPs are connected directly to the Finchley, only the tPoPs compete for the bandwidth. If you are using a tPoP you will have to see if the saving of the local call outweighs the degradation of the service. As I've stated in the case of the Energis vPoPs, this situation is eased as all the modems are actually connected directly to the Finchley network. Therefore if you have the choice between a local vPoP or tPoP I would suggest that you use the vPoP which should increase your throughput. The numbers for these lines are available in Welcome.txt (on ftp.demon.co.uk) 2. MODEM.TXT == ========= Advice: Read MODEM.TXT (this advice is traditionally chanted in unison)! How: Demon maintain and publish a related document to this one called "MODEM.TXT" which you should find in the KA9Q distribution ZIPfile, or on ftp.demon.co.uk as /pub/doc/Modem.txt (the M is a capital). Why: MODEM.TXT explains about using proper cables to your modem and the basics of setting up your modem for use with KA9Q. Suggestions that it fully explains the meaning of life are unfounded, but it (like the other Demon produced documentation) does cover a lot of ground and reading it is to be highly recommended. The purpose of MODEM.TXT is to get you connected and data flowing. This document on the other hand is intended to get that data flowing at an optimum rate. If you post to the demon.ip.support.* newsgroups and cannot say that you have read MODEM.TXT (and of course this document TUNING.FAQ) then you are wasting everyone's time, including your own. People may well point this out to you by email; more or less politely. 3. MNP5 and other compression == ========================== Advice: Configure your modem to use V42bis not MNP5. If you only have MNP5 then use it only for text transfers How: RTFM! Sorry, but the AT commands for configuring your modem are not standardised across different models. MODEM.TXT contains some set-up strings which may help you. Why: MNP5 is a data compression scheme which modems use whilst the data is transferred across the phone link. Unfortunately, although it will compress text fairly well, and is said to be especially good at UUENCODED text, it can (and does!) make ZIP files bigger! V42bis is a compression scheme which is smart enough to quit if it is making things worse. If your modem does V42bis then use this and turn off MNP5. Otherwise you should turn off MNP5 unless you know that you will only be transferring text. Some people go as far as to arrange two separate set-ups and they log off and redial with the other set-up if they are going to FTP some ZIPs rather than collecting their newsfeed. 4. Hardware overruns == ================= Advice: You MUST try to eliminate hardware overruns. How: You can tell if you have hardware overruns by using the KA9Q asystat command (just type it at the NET prompt). If 'hw over' is non-zero then you have a problem. If 'sw over' is non-zero then you also have a problem. For more about asystat in general, and software overruns in particular, see section 9. To fix a hardware overrun problem: Slow down the link to the modem see section 5 Try using a Windows DOS box see section 6 Try not using a Windows DOS box (!) see section 6 Get rid of TSRs, SMARTDRV &c see section 7 Buy a 16550A see section 8 Why: If you are getting hardware overruns then your throughput will suffer! Because of the nature of the TCP/IP link you will suffer a lot more than you would on normal comms links to Bulletin boards. Thus a set-up which works marginally to, say, CIX can work very badly indeed to Demon. Your modem is generating characters in an unstoppable sort of way, though at a maximum fixed rate (9600, 19200, 38400 etc.). When the characters turn up at the PC end of the cable they must be recorded into the PC memory before the next character turns up. If they are not "read" and transferred to memory then they will be lost. When the standard 8250 serial chip "interrupts" the PC to tell it that a character is ready then the PC has just one character time to read it before the next one appears and replaces it. (Of course if the next character is late arriving because the link is not fully utilised then the PC has correspondingly longer to react.) If the PC loses a character then the entire packet it formed part of will have to be sent again. Packets may well be 1500 bytes long so this can take some time. Further, because of the way the TCP/IP protocols work the re-sending may not start until after a non-trivial timeout has elapsed. Worse, if you are receiving data from a long way away then at any given time there is a lot of data "in the pipeline" coming to you. When the other end realises that you've lost some data it will resend that data and continue on from there. Naturally KA9Q will hang onto the good data which turns up after the damaged packet and thus once the retransmission starts and the missing packet arrives it can (and does) acknowledge everything it has. However, by the time the other end receives this acknowledgement the pipeline may be refilled and thus, despite KA9Qs best efforts, you will still receive a lot of duplicate data, which takes time to be transferred to your PC and then discarded. All of this means that hardware overruns have a significant effect on overall throughput, and their elimination is extremely important. 5. How fast should I run the link to my modem ? == ============================================ Advice: Modern modems compress the data they send across the phone line, so to keep up you need to run the serial link to them quite a bit faster than the speed they have apparently connected at to the other end. Use the fastest possible serial link speed to connect to your modem PROVIDED that you do not get hardware overruns. 19200 bps is too slow for a 14.4K connection. How: The serial link speed is configured by the DIS front-end program, [D A] (your chosen setting eventually ends up in the ppp attach command in autoexec.net in the main KA9Q directory). e.g. attach asy 0x3e8 4 ppp sl0 4096 1500 38400 ----- Note: The letters "AT" at the start of each modem command form a (carefully chosen) bit pattern which allows your modem to automatically detect the serial link speed. Thus modems do not usually need any reconfiguring when you try different speeds. # It should also be noted that the current version of KA9Q cannot handle this throughput being set to 115000 due to a bug which should be fixed in the next release. Why: There are two speeds involved in communications. First there is the speed at which bytes of data are transferred to and from the modem. This will be either down a cable to an external modem, or in the case of an internal modem, the speed at which the interface chips pass data to the rest of the card. This document calls this the "serial link speed", but elsewhere you may see it called the DCE/DTE speed. The second speed is the connection speed across the telephone wires. Not all that many years ago modems ran at 1200 baud, and you used 1200 bps serial links to them. Modern modems use complicated encoding methods so that transitions on the wire no longer directly correspond to bits of data (this is the distinction which professionals and pedants like to make between bps and baud). Furthermore, if requested, the data itself is compressed (using MNP5, V42bis etc.) before it is transmitted and it is then re-expanded at the other end. As a result you can regularly get text transferred at 3000 or more bytes per second down a 14.4K link. (In fact, sequences of identical bytes can be compressed almost out of existence. The protocol headers on TCP/IP packets prevent this occurring, so it is very unusual to see speeds much above 3200 bytes per second). Since there are 10 bits per byte on a serial link, using 19200 bps will only allow you to transfer 1920 bytes per second. Obviously if you have a 14.4K modem which can move over 3000 bytes per second of textual data this can create a bottleneck. Using 38400 bps will almost invariably make the phone link the limiting factor. At the other end Demon connect their computers to their modems at 38400, thus overall (in a steady state) 38400 bps is sufficient. But a steady-state analysis is not the end of the story, since your PC might get a little behind in servicing the serial port (because of writing to disk for example). Therefore it is theoretically helpful to run at 57600 so that your PC can catch up, should it ever get behind in this way. However, since 38400 already exceeds the transfer rates that you will get on real data (which never contains indefinite runs of identical characters), then if your modem or serial card will only do 38400 this is not a matter to cause you much concern. At faster speeds your PC has less time to deal with incoming characters before the next one appears. For example, 38400 produces characters four times as fast as 9600. This can produce hardware overruns - which as discussed above can severely degrade throughput. If you get hardware overruns at 57600 then try 38400 since this will be just as good in practice. Similarly, if you get overruns at 38400 then try 19200. Some people state that a few hardware overruns at 38400 gives them better overall performance than no hardware overruns at 19200. The trade-off will depend upon how soon the TCP/IP link resends data and how quickly your acknowledgements reach the other end and this will in turn depend upon how much Internet there is between you and the data sender. It would naturally be better to try the other advice in this document and achieve 38400 and NO overruns! Of course not everyone has a 14.4K or 28.8k modem. Demon do not let you connect at 1200 because you cannot get packets back and forth quickly enough and some protocols time out. Many people use 2400 or 9600 bps modems ... they may well find that there is no advantage in increasing serial link speeds past 4800 or 9600. However, provided that there are no hardware overruns there should be no disadvantage to higher speeds, so use them if you can. # Demon are now in the process of updating all their modems to v34. All the vPoPs have now been upgraded and work on the the older PoPs is being planned. Demon have also increased the speed of their terminal servers from 38400 to 115,000. This should give a considerable increase in performance if you have a V34 modem. 6. Does Windows affect link speeds ? == ================================= Advice: Try running KA9Q in a full screen Windows DOS session ! (or possibly, avoid running KA9Q in a Windows DOS session !!) How: Provided your SYSTEM.INI file correctly lists your hardware you should have no problems using serial ports from a DOS session. Beware of serial mice ... in the usual configuration of a PC COM1 and 3 share the same IRQ (as do 2 and 4). This might not show up if you don't load a mouse driver except within Windows. Why: Under Windows real hardware devices are handled by device drivers, and programs like KA9Q do not actually access the hardware directly. In principle this "virtualisation" slows things down so DOS sessions should be worse than not using Windows. However, because Windows device drivers are well integrated into the system, you may find you can use faster speeds without getting hardware overruns. Even the slowest PCs are quite fast enough to handle 3000 interrupts a second from a serial port. However, besides processing the incoming data the information is also stored on disk, and text is written to the screen to keep the user appraised of progress. Unhappily, whilst doing this DOS and the PC BIOS can lock out interrupts for relatively long periods, and so the PC is not able to break away from another task and service the serial port. Thus you can get hardware overruns. Avoiding any quasi-religious discussion of the merits of alternative operating systems, we can note that Windows has avoided these problems with DOS and the BIOS (in the jargon, it can handle short crisis time devices), and so you probably get fewer overruns than from raw DOS. The word "probably" in the last sentence was chosen carefully. Certainly many machines do run better, but some people do worse in Windows DOS sessions than otherwise, though seldom so much worse as to make it worth changing. Fast machines and SCSI drives seem to help you do better under Windows than DOS. Slow machines seem to work better under DOS than Windows. The only plausible advice seems to be "try it and see". Running in a Windows DOS session is much less likely to be beneficial if you are not in full screen mode. When the session is made a window on the desktop there is a significant overhead involved in rolling text up. This can cause data loss. If you change away to another window then how much attention KA9Q gets will depend upon the other program's ability to share machine time, and on the various priorities of the tasks you are running. 7. Eliminate SMARTDRV, TSRs etc. == ============================= Advice: If you are getting hardware overruns then experiment in running without DBLSPACE, SMARTDRV, MIRROR or inessential TSRs. Even if you don't remove these utilities, some configurations are worse than others, so parameter twiddling may help. How: RTFM (for DOS or the TSR) Why: As indicated in the previous section, you get overruns when your PC is not informed (by an interrupt) of an incoming character, until it is too late. Disk compression and disk caching software can disable interrupts for long periods and thus cause problems. In general, the faster your machine the faster it will be executing these critical sections, and so the less likely you are to have problems. DBLSPACE seems to have caused a lot of problems to people who have tried to use it. A common workaround is to arrange for the incoming newsfeed (in _\spool\articles) to be on an uncompressed disk, whereas the newsbase (_\spool\news\newsbase) is kept on a compressed area since it is only used off-line when disabling interrupts is of little consequence. If you want to change the incoming temporary directory then either edit the configuration files directly, or use the DIS front-end [E J A] to change the line starting "newsdir"; then [D F A] to change the line starting "nntp dir" to correspond. You then need to edit DEMON.BAT by hand; in the section starting :net change the directory tested for BATCH.TXT. SMARTDRV has also caused problems, though many people seem to feel that it can improve throughput if you can make a big cache with delayed writes. Small caches with delayed writes seem to cause problems. Big caches without delayed writes generally seem to be OK. These results may just be caused by the bigger caches making it less likely that *any* disk transfers are needed. Similarly, since most data is incoming, turning off write caching will mean that a disk cache has little enough to do whilst you are on-line. This is obviously an area which requires experimentation to determine the best solution for your configuration. You could try editing the DEMON.BAT file to add extra enable/disable commands before and after running KA9Q. 8. Serial chips == ============ Advice: Use a 16550A serial chip If you fit a 16550A you can probably ignore all the other advice in this document about eliminating hardware overruns, because you will find that no matter what, none will occur. Or to put it another way, if 16550As were free, sections 4..8 of this document could be discarded altogether. How: Most PCs are still shipped with 8250 serial chips (UARTs). If socketed they can be directly replaced by a 16550A. If soldered in, or if your serial card is a modern highly integrated device (doing parallel, disks, washing machine etc.) then you will need to add another serial card. If you don't have a spare slot then you'll need to buy a multifunction card with a 16550A on it. A quick glance at MicroMart (Thursdays, 60p) will yield literally dozens of suppliers of 16550A boards and chips. They change so fast that specific recommendations are impossible, but as a guide you should pay no more than 20 pounds (+VAT & postage) for a serial card and a fiver or more less than this for just the chip on its own. Fans, and one-stop shoppers, can also purchase these items directly from Demon, but although service and support may be better, their prices are less keen. If you aren't sure what sort of chip you have then the Microsoft supplied program MSD.EXE (in your DOS or Windows directory) can probably tell you. Run the program outside of Windows for the most reliable answer. However MSD can become confused by some multi- function cards, and if you have an unusual configuration it can fail to identify which port is which! Numerous other utilities exist which will check out your serial cards. Try ftp.demon.co.uk for /pub/simtel20/msdos/modem/modemd52.zip. Also, of course, KA9Q will tell you the details of the port which it is using. See the description of asystat in the next section. In passing, it should be noted that there are a fair number of alternative serial chips, as well as dual port versions of the 16550A. The 16550 is not suitable (it contains a FIFO, but it does not work properly). KA9Q correctly detects all the chip variants whose FIFO can be used. Christian Blum has collated an excellent FAQ called The_Serial_Port which covers the various alternative chips in detail. You can find this at pfsparc02.phil15.uni-sb.de in the directory /pub/E-Technik/afd. We will not repeat all of the information here since: * the 16550 has not been manufactured for years so now you are unlikely to be offered anything other than a proper 16550A, * the various other letters at the start and end of the chip part number are manufacturer specific, or just tell you whether it is NMOS or CMOS. so just order a "16550A" card or chip, and of course if it is a replacement then tell the supplier what the current chip designation is. In the unlikely event you get the wrong device then the Sale of Goods Act will protect your investment :-) Explanation: The important difference between an 8250 and a 16550A is that the latter contains a 16 byte first-in first-out ("FIFO") buffer. This means that when bytes turn up on the serial link the "crisis time" before they are overwritten by further characters is significantly extended. As previously mentioned, when a PC is doing nothing but processing serial data there is no problem in it keeping up, and it turns out that the modest increase in buffering provided by a 16550A is quite sufficient for current applications and modem speeds. In fact the buffering is more than is needed in practice, so that a secondary benefit is possible. Instead of interrupting as soon as a character turns up (and giving the PC 15 or so more character times to respond) the chip can be configured by software to buffer several characters before interrupting at all (whilst still leaving several spare slots in the FIFO for further characters). This means that the PC is interrupted less often, and this can improve performance slightly. Naturally, there is a scheme whereby if no further characters seem to be appearing, then an interrupt is generated for the partially filled buffer - you can detect this happening with the asystat command. The reduction of interrupt load of a 16550A is of particular benefit to Windows which you will recall must "virtualise" the hardware device. This is all overhead, and the less the better. Sadly the standard drivers shipped by Microsoft take limited advantage of the 16550A. There are some third-party alternatives such as the Cybercom driver on ftp.demon.co.uk in /pub/ibmpc/windows/utilities/cyberc.zip. It uses different settings to make better use of the FIFO buffers. The difference is said to be greatest on Windows 3.0, and hard to measure on 3.11. The bottom line is that a 16550A is a Good Thing and as many people know, not only in theory! As modem speeds increase it will become ever more necessary. Further, it can help a bit even if you weren't getting hardware overruns - and it might let you put back some of your TSRs (DBLSPACE, SMARTDRV or whatever). Of course, if you want to be really fancy you can buy proprietary enhanced serial interfaces or even serial boards with 16K buffers on them... but not for the same price as 16550As! 9. The KA9Q asystat command == ======================== Advice: Use the asystat command to check for hardware and software overruns. Discussion: When you type asystat at the net> prompt KA9Q will tell you a lot of useful low level information about how the serial link has been performing so far this session. e.g. sl0: [NS16550A] [trigger 0x7e] 38400 bps RX: x int, x chr, x hw over, x hw hi, x fifo TO, x sw over, x sw hi TX: x int, x chr, x q, x MS int, x THRE TO The first line tells you about the hardware configuration: 'sl0' is the mnemonic name for serial link interface zero. '[NS16550A]' shows that KA9Q has detected that a 16550A is fitted and it is using the FIFOs. It will be absent if you have some lesser chip. '[trigger 0x7e]' is to do with the protocol used, and is not of general interest. '38400bps' is the serial link speed between the modem and PC. The first line will also tell you if CTS flow control and/or RLSD (carrier detect) line control is enabled. The second line gives statistics for received (RX) information. 'int' is the total number of interrupts which have occurred. 'chr' is the total number of characters received. These numbers show you how busy your link has been. On a 16550A (though not necessarily on the Hayes ESP interface) you will see that 'chr' is much more than 'int' because characters are transferred in batches. These are raw numbers, including all protocol headers, escape characters and any duplicate data. 'hw over' is the total number of hardware overruns which have occurred. ie this is the number of individual characters which have been lost. FOR BEST RESULTS THIS VALUE SHOULD BE ZERO! 'hw hi' is the highest number of characters read during a single interrupt. It is reset to zero every time you do an asystat command. As this number approaches the buffer size in your chip it indicates how much of a risk you are running of getting a 'hw over'. If you run under Windows you may well see values of 30 or more here. This is showing you how many characters the device driver is buffering, rather than the size of the buffer in the chip itself. 'fifo TO' is only reported for 16550As. It is the number of times interrupts were generated because characters were in the buffer but no more were arriving. TO stands for time out. 'sw over' is the number of times that the KA9Q buffers have overflowed. Just as with hardware overflows this is bad news. FOR BEST RESULTS THIS VALUE SHOULD BE ZERO! If you are getting 'sw over's then you need to increase the buffer sizes by altering the attach command to allocate a larger input "ring buffer": e.g. attach asy 0x3e8 4 ppp sl0 4096 1500 38400 ---- Try 8192 or even 16384. Actually, any value you use will be taken to the nearest multiple of 8, so you don't need these very "techie" powers of 2 for the size, so feel free to try 10000, 11000 etc. 'sw hi' is the "high water mark" showing the maximum space ever used within the KA9Q buffer. If this value is always much less than the buffer size then you could safely reduce the buffer size, and free up memory for other purposes. This value is reset to zero by the asystat command. The third line gives statistics for transmitted (TX) information. 'int' is the total number of interrupts which have occurred. 'chr' is the total number of characters transmitted. 'q' is the total characters currently queued for sending. 'MS int' counts modem status interrupts. You'll usually see a handful of these, corresponding to your modem managing to CONNECT. If you are using CTS or RLSD line control then this number will tell you how often this is occurring. 'THRE TO' relates to transmit interrupt timeouts. It is not of general interest. 10. Internal modems == =============== All of the advice given in sections 4..7 and 9 applies equally well to internal modems. An external modem lives at the other end of a cable from the computer. As characters arrive over the phone line it will send them down the cable to the computer whether or not the last one has been dealt with yet. An internal modem is directly connected to the serial port hardware within the computer, in fact they will be on the same board if not the same chip. Thus an internal modem has access to the serial chip and can therefore, in principle, "know" whether or not the computer has processed the last character yet. This allows internal modems to use their own buffering system to avoid "overruns", should the computer occasionally be slow at processing incoming characters. For their serial port hardware internal modems may actually use an 8250 or a 16450 chip (much the same as far as this document is concerned), or more likely they will contain some custom chips which emulate one or the other. Either way, for the reasons just mentioned, they will not generally exhibit the same sort of overrun problems that an external modem with a real 8250 would suffer from. Thus you should not be concerned if your modem does not contain a 16550A. To be perfectionist, everything else being equal, then you would prefer an internal modem to emulate a 16550A rather an 8250 because of the lower interrupt load. However, provided the buffering is adequate, the difference in KA9Q performance caused by fewer interrupts may be hard to measure. 11. IP fragmentation === ================= Advice: Make sure that incoming packets are not being fragmented. How to detect fragmentation: Use the KA9Q "ip status" command. Look at the variable (14) called ipReasmReqds. If it is not zero then you are getting fragmented packets and this must be corrected. The other statistics count IP packets and most of the other fragmentation values (the ones towards the end of the list) should also be zero except ipReasmTimeout which will be 30. This is the time KA9Q waits for the rest of a fragmented packet to appear and hence it is correct that it is not zero! Background - what is fragmentation: When data is sent from machine to machine over a TCP/IP link it is parcelled up into packets. The end machines agree the size to be used which is called the maximum segment size (MSS). The data has 40 bytes of TCP/IP header added and is then placed inside of a hardware datagram, on a Ethernet, X25 network or whatever is used to move the data across the Internet, possibly through 30 or 40 hops from the other side of the planet to your machine. If at any stage the data will not fit into the datagram for the next hop it will be split up (fragmented) and the fragments travel onward to be reassembled within KA9Q. Since the fragments each have their own header there is an extra overhead associated with fragmentation. You might think that this overhead was the difference between (header(40) + data(n)) and (40 + n/2 + 40 + n/2 = 80+n) but it is in fact a great deal worse than this... ... On PPP connections (and indeed on SLIP as well) the packet headers are usually sent using "Van Jacobson" (VJ) compression. This very clever scheme means that headers are typically transferred in only 5 bytes or so instead of the usual 40. However, fragmented packets are never compressed (since in a well-ordered network they should never occur). Thus, in the example, the difference fragmentation makes is between (5 + n) and (80 + n) [approximately]. On a typical connection to Demon using the standard KA9Q configuration fragmentation will degrade your performance about EIGHT PERCENT. This is very significant, and is well worth avoiding. There is a scheme which is being introduced across the Internet to allow machines to dynamically determine the MTU (maximum transfer unit) between them. This "path MTU discovery" is used to ensure that datagrams are never bigger than the smallest size on any link they must traverse. Sadly, many machines do not use this scheme, so human intervention is required to set the values correctly. More advice: Use an MSS of 536. Then set the KA9Q ppp link parameters to use MSS+40 (ie 576) as the datagram size (the MTU). This in most cases is the default setting. You can do about 0.6% better than this in one special case... If you are connecting ONLY to Finchley (not to any other PoPs) AND you will not be using the trans-Atlantic link (ie you are only going to receive email and Usenet news, or use the local ftp.demon.co.uk) THEN (and only then) you can use an MSS of 1460. Set the ppp link parameters to MSS+40 (ie 1500). You MUST also change your login script to specify the protocol as "PPP,mru=1500" rather than just PPP. (mru is an equivalent acronym for MTU!). Remove the mru=1500 if you change away from 1500::1460 otherwise KA9Q will fail to connect. Note: In the past Demon used different datagram sizes on their links. In that different world, different settings were optimal. Any previous, now out of date, advice should be ignored! Always using the 576::536 (MTU::MSS) setting will cost you less than a 1% overhead compared with "optimum" 1500::1460 value for Finchley only usage. Using 1500::1460 to any other PoP, or across the Atlantic will immediately risk an 8% degradation. So unless you are very sure that your usage fits the special case you should pick 576::536. If you regularly transfer files from non-Demon server machines then you should check for IP fragmentation. Although 576::536 is a good setting for most places on the Internet there is no guarantee that smaller values are not being used on some of the links the packets traverse. If you use telnet a lot then more advice is given below. How: Decide if you are going to use an MSS of 1460 or 536 (or perhaps less, see later advice). Now edit the autoexec.net file in the main KA9Q directory. You change the lines: attach asy 0x3e8 4 ppp sl0 4096 1500 38400 ---- tcp mss 1460 ---- ppp sl0 lcp local mru 1500 ---- ppp sl0 lcp remote mru 1500 ---- Note that sl0 is "s" "el" "zero" (serial line 0); an error-prone choice of identifier! You change the 1460 value to your chosen MSS (usually 536) and the 1500 value to MSS + 40 (usually 576). Also, if you are not using 576::536 (for example if you choose to use 1500::1460) then you need to edit the dialler sequences to respond "ppp,mru=1500" to the "Protocol" prompt. You can do this from the DIS front end (option D, option A, f5) or by editing the DIALER file directly. The mru= value should correspond to your MSS + 40 value. There is no harm in using mru=576, but it is not necessary. Why: The situation with remote tPoPs (Warrington, Reading etc.) is simplest to explain. Demon have configured the link to and from Finchley to use 576 byte datagrams, ie there is room for 40 bytes of TCP/IP header and 536 bytes of data. They have done this because they want short packets travelling the link, so that when you call the PoP the login sequence can transfer information back and forth without having to wait for very long packets to finish. This 576 value is fixed, so you must set the MSS to 536 or less (and therefore set the total packet size in the ppp parameters to 576 or less). As the vPoPs are based at the Finchley Network Centre they should have the same performance as the original Finchely tPoP. Similarly, the transatlantic link is configured to use 576 byte datagrams (again to improve general responsiveness). Any packets larger than 576 are fragmented before they can be carried across it. The routers at Finchley have a default configuration of 576, so there is no problem in connecting to them at this size. However, Demon's Ethernet LAN uses 1500 byte datagrams, and it is possible to configure the Finchley routers to use 1500 byte datagrams on the phone link by adding "mru=1500" to the login sequence. You then set the MSS to 1460 and configure the KA9Q ppp link to 1500. Provided your traffic just flows across the Demon LAN (ie is just incoming email, Usenet news and local FTP) then it will not be fragmented. In theory, you would only need to alter the KA9Q PPP settings, and the magic of the PPP protocol will correctly reconfigure the router at the other end of the link. However, the routers at Finchley have been configured to 576 in a "broken" way (ie not conforming to the RFCs) and so if you set 1500::1460 (the default KA9Q configuration!) then the link will actually be configured to run at 576::1460 ie with fragmentation occurring and 8% less throughput than you might expect. Setting mru=1500 stops this problem occurring. Configuring to 576::536 also prevents the problem. When the ramifications of the "broken" routers on IP fragmentation were first beginning to be understood it was found that you could get 1500::1460 by asking for 1501::1460 (!) and for a while setting 1501 was the best available advice. This still works, but the advice above is simpler to understand and apply. Remember that all of the numbers given above (except for 1501::1460) correctly use the relationship "n"::"n-40". The values explicitly EXCLUDE the PPP header byte and various other red tape associated with using PPP, which can be ignored when determining the numbers in this section. As it happens, there is a bug in the PPP code of KA9Q (reported, but not yet corrected). This means that negotiating a link value which is smaller than the link value at the Demon end will fail, and you will get to the HELLO prompt but no further. Negotiating to higher values does not seem to suffer from this problem ... except for the problem of negotiating up to 1500 discussed above. The simplest way of avoiding any problems is to always set the mru= value on the login line to the same as the value you want KA9Q to negotiate. Thus if you wanted to use, say, a small mss value like 266 then you must set the ppp negotiation to 306 (ie 266+40) and you should set the login line sent by the dialler to be "PPP,mru=306". Even more advice: If you regularly use telnet at the same time as an NNTP news transfer (or FTP file transfers) then consider a smaller MSS. How: As explained above, set the tcp mss command to "n" and the attach and two ppp setup commands to n+40. You must also set the dialler login line to specify mru= the n+40 value. Why: When you are using the telnet protocol you are sending very small packets (with just a character or two in each). In the absence of fancy priority queuing these packets must wait their turn behind the big 1500 byte packets. Since 1500 bytes of FTP will take about a second to come over the link you can see that this can make telnet response rather jerky. Smaller packets mean that the telnet information can flow more smoothly. This is in fact much the same scenario as the speeding up of remote PoP logins discussed above. There is obviously some overhead in using smaller packets. However because the headers are compressed so much, it is not very high. Van Jacobson's RFC on compression recommends choosing 240 at 9600 bps. At this size, you can send 1460 bytes in just over 6 packets. Thus the extra overhead is about 25 bytes per 1500. ie even with quite small packets the "cost" is only about 1.6%. Note that if you are *only* doing telnet then the only packets for transfer will be telnet packets and so you will see no difference in response no matter what transfer sizes you use. See also section 15 for other reasons for slow telnet response. 12. NNTP request queues === =================== Some of the throughput problems with Usenet news are caused by delays at the Demon NNTP server. It is possible to improve the flow by an "nntp batch on 12" command in autoexec.net. This issues requests for articles 12 ahead so that the overworked news machine is able to start on fetching further articles whilst KA9Q is still receiving a previous one. You can change this setting from the DIS front end program [D A f6]. [The value 12 is chosen random(ish)ly. It has been suggested that even with higher values the flow is still more "jerky" than might be desirable.] 13. HISTORY === ======= Advice: If you are getting slow transfer times then slim down your HISTORY file (by hand or by expiring some news). Slow transfers are a problem that comes suddenly upon people after days or weeks of acceptable performance. It only affects NEWS; email and FTP will still work at full speed. How: Besides discarding or archiving old news the EXPIRE program will also trim your HISTORY file (kept in _\spool\news) according to the criteria set in EXPIRE.DAT (same place). The relevant command is: [tail] 1000 where 1000 is the number of news article identifiers to be retained in the HISTORY file. A sensible value is just over one day's worth of articles, but anything under 2000 is likely to be OK. Note: If you run expire directly (as in "expire 10") then it will not use EXPIRE.DAT and it will not reduce the size of your HISTORY file. You will need to take other steps to do this. Why: The purpose of the HISTORY file is to record article identifiers which have already been downloaded to your system, so that even if you are offered them again you will not refetch them, but will instead see them reported as duplicates. KA9Q keeps this list of identifiers in memory if possible, but failing that it has to read the list off disk in order to check. It is these disk transfers which ruin the performance. You can of course use different values than 1000 for tail, indeed some people use 0, and can thus avoid using expire but just delete the HISTORY file altogether. If you use 0 then any duplicate news will be downloaded, but the SNews unbatch program will still discard the duplicates using the HISTORY.SNW file (whose contents are controlled by the [life] parameter). It is usual to be offered a few duplicates at the start of each NNTP session (because of the way the "last fetched news" time is handled). Therefore, if you use [tail] 0 you must be prepared to waste time downloading a few articles which will be discarded by unbatch. A higher setting is therefore to be recommended. Note: Some people have managed to damage their EXPIRE.DAT so that the [tail] command is missing altogether. Of course, running expire in this state will not reduce the size of the HISTORY file. 14. PPP versus SLIP === =============== Advice: Use PPP. How: Send "PPP" to the protocol prompt. This is the default set into the DIS front end program. (Better, usually is "PPP,mru=1500" or whatever size you require; see section 11 above.) Discussion: Demon recommend using PPP for connections with KA9Q, and the latest versions will have had SLIP removed. There are better diagnostic tools at the Demon end for PPP problems, and this doubtless contributes to their preference for it. However, Windows applications working through Winsock usually seem to use SLIP instead. Sometimes people ask why... SLIP is a fairly simple scheme for sending TCP/IP packets down a serial wire. The code is trivial (assuming you are happy about interrupts and serial chips and such). Compressing SLIP headers (RFC1144) is not entirely trivial, but is not a vast amount of code and improves telnet responsiveness especially (and reduces link traffic generally). Some example code is in the RFC so it is pretty easy to add to an existing program. This means that people can easily support SLIP - so they do. Some people use the name CSLIP for SLIP- plus-header-compression. This document does not bother to distinguish, because there are very few implementations of SLIP which are not CSLIP as well. PPP is a rather fancier animal altogether. It will handle multiple protocols down the same wire (which is not very relevant in this context). It can also deal with links which use software flow control or which cannot transmit some characters (which is also not very relevant). With suitable compression negotiated you get much the same overhead per packet as with SLIP (it rather depends quite what you send!). Since it is a bigger (and newer (the latest RFC came out just before Xmas)) protocol there are rather fewer implementations around, however KA9Q does have PPP built in. Winsock protocol stacks often use packet drivers for their network connections. The Crynwr set of freely available packet drivers does not at present contain a PPP driver. There are in fact very few PPP packet drivers around (which you don't have to pay for) and their reputation for using 8250s (as opposed to 16550As) is not very good. The Trumpet shareware Winsock stack can either use a packet driver or its own internal SLIP. The very latest beta versions of the Trumpet stack now support PPP and some people have used this successfully (and others have not). All of this means that at present most Winsock users are connecting with SLIP. The received wisdom is that PPP is a teeny bit faster in practice than SLIP, but only by a little bit. Since they both tend to add the same protocol overheads of single bytes each end of a packet, and can both use header compression the transfer rates will depend upon horrible practical issues of error recovery which will depend on the sort of corruption you get on your link; whether your error correcting modem actually does correct your errors; and how many data octets [ that's bytes :-) ] need escaping. This similarity in performance means that other issues elsewhere in the software can easily swamp the differences between competent implementations of either protocol. So the bottom line is that you should use PPP when you can, and SLIP if you can't, and it doesn't make a lot of difference either way. 15. Telnet speed === ============ Telnet responsiveness was discussed above in the section on IP fragmentation. However, you should also be aware that you can get very "soggy" links when the remote end is echoing what you type. The reason is simply that the other machine is a long way away. It's typically 300ms to Finchley, 800ms to the US and 1.6 seconds or more to Europe (because European traffic crosses the Atlantic twice for reasons more to do with politics than technical issues). You can't change these routes - you're stuck with them - and so echoes from the remote machine can take a long time. Also, when dealing with European machines, you will find that the infrastructure is slower and many more machines are connected by fairly slow and congested links. You can use "ping" to find out how far away the remote machine is. Try "ping xx.xx.xx" when you've gone "telnet xx.xx.xx" to see how far away the machine is. If you're interested in the path taken there then try "hop check xx.xx.xx" and you'll see the little tour of the Eastern US states that most traffic takes. When using "ping" you must remember that the packets it is sending are queued for transmission in the normal way. Thus if you "ping" during news transfer you will get higher values than otherwise. Since this is pretty much what happens to telnet traffic, this may contribute to the information you need to tune your packet sizes. 16. Other improvements === ================== Whilst receiving news and email the incoming data must eventually be written to disk. The usual suggestions for increasing disk performance apply : defragment the drive, set BUFFERS= to a sensible value, use SMARTDRV, use a RAMDISK etc. All of these are Good Things provided, as discussed extensively above, you check that they don't introduce hardware overruns. Several people with more than one drive have reported improvements from ensuring that the incoming _\spool\articles\BATCH.TXT file is placed on the faster drive. Finally, it has been suggested that provided your modem has software flow control disabled (ie you can send _Q and _S (XON, XOFF) as data) then there is little point in having the ppp protocol escape these values. This change would make a marginal difference to ZIP transfer speeds, but none at all to news and other text transfers. Has anyone tried this ? 17. OS/2 == ==== All multi-tasking operating systems use some of the available machine performance to do their magic, so if you are short of machine power then OS/2 (or indeed NT or plain Windows) will use machine cycles which would otherwise be used to execute KA9Q. However, most modern machines are fast enough that you are in practice unaffected by the overhead, and you are actually limited by the serial link speed. On such a machine (like a 25 MHz 486) you will see no difference between, for example, KA9Q for OS/2, KA9Q in an OS/2 DOS box or KA9Q running under real DOS. On a more modest PC (like a 20MHz 386SX) you will find that using OS/2 will slow down News collection considerably (1400 cps rather than 2700, say), but that FTP speeds will be almost unaffected. You have the option of running two different flavours of KA9Q under OS/2: * DOS KA9Q v2.16 in an OS/2 DOS session earlier versions do not have important buffering fixes. * OS/2 KA9Q v2.17 (OS/2 version 2.40) running as a PM application and including Archie and Gopher clients. again, avoid earlier versions because OS/2 version 2.40 fixed a long- time bug that caused lockups. The latest version is on ftp.demon.co.uk in /pub/os2/netos2. This version is exempt from advice in earlier versions of this document to specially boost the process priorities. You should install Ray Gwinn's shareware SIO/VSIO serial port drivers to get a virtual buffered serial port with a 4096 character buffer. These drivers provide most benefit to DOS comms applications running under OS/2. Find these on ftp.demon.co.uk in /pub/os2/netos2 as the file sioXXXX.zip, where XXXX is the version number. Don't forget to register them. A 16550A is strongly recommended for use with OS/2, although the SIO/VSIO drivers can provide near 16550 performance on machines with unbuffered UARTs. To set the "performance baseline" for your machine you could try booting "real DOS" and running DOS KA9Q v2.16. (This will only be possible if you have dual boot or boot manager installed and KA9Q is running from a FAT partition.) You can use this performance level to assess how well your OS/2 setup is doing. Note that the OS/2 and DOS versions of KA9Q will happily share the same configuration files, so swapping around to experiment is relatively easy. 18. The Authors === =========== This FAQ is maintained by Richard Palmer (Tuning@blackbrd.demon.co.uk It is based on the Tuning FAQ by Richard Clayton and Phineas R John. Many of the ideas and advice that have been culled from the demon.ip.support.* newsgroups. I hope that the original authors will be pleased to see their ideas here, even if their names are absent. 19. Disclaimer === ========== Although whilst writing this document I have tried to be accurate, and to give good advice, I have not spent the time or effort on it that would be necessary for it to be in any sense authoritative. In particular the efforts I have made fall short of the sort of standard which would be expected from us in our professional capacities. You follow any advice within this document entirely at your own risk. Take a note of settings before you change them, and always take a back up copy of data which is of value to you. Naturally I would be pleased to receive email with corrections and suggestions for improvements and alterations. Please write to: Tuning@blackbrd.demon.co.uk --------------------------------------------------------------------------- Copyright 1995 Richard J Palmer (Tuning@blackbrd.demon.co.uk) Original FAQ Copyright Richard Clayton & Phineas R John. This file may be freely distributed provided that it remains unedited from its current form. The latest version is posted regularly to the newsgroup demon.ip.support.pc. It is available by FTP from ftp.demon.co.uk as pub/doc/ka9q/Tuning.faq or can be obtained upon email request to Tuning@blackbrd.demon.co.uk end of TUNING.FAQ Issue 29 10th July 1995 ---------------------------------------------------------------------------